Asymptotic results on codes for symmetric, unidirectional, and asymmetric error control

2073 The bound can also be adapted to continuous alphabets by replacing the probability distribution p(.) by a density, the cardinality lB,(p)l by a volume, and the entropy H (p) by the corresponding differential entropy. With these substitutions-and provided that a density p(.) of the form (3) and satisfying (4) exists-the nonasymptotic part of the first proof, and thus the bound (2), is still valid. We conclude with the following example due to G. D. Forney, Jr., (private communication). Example: Let A be the real line with weight w (a) = a 2 ; then B,J p) is the n-dimensional sphere (ball) of radius 6 around the origin. The probability density p (.) is Gaussian with variance p, whose differential entropy is log, fi. According to (the continuous version of) (21, the volume of B,(p) is upper bounded by (2 ~ e p) " ' ~. The comparison of this bound, for n = 2m, with the exact formula (2 m p ~) ~ / m ! for the volume yields the Stirling-type bound m ! 2 (m / e) " , derived purely from information theory and geometry. (The Stirling approximation is m! = f i z (m / e) m .) APPENDIX PROOF OF THE PROPOSITION To simplify notation, we write w, and p , instead of w(al) and p(al), respectively. All logarithms are to the base 2. are the elements of A that have minimal weight. For A = 0, p(.) is uniform over A , and thus E [ w ] = W and H (p) = log IAl. The limits as A-+ K-of p (.) is the distribution p , = l / m for 1 I i I m and p , = 0 otherwise, which makes it clear that 1imA+= E [ w ] = w,,, and limA+x H (p) = log m. We next show that (d / d A) E [ w ] < 0 for all A. Let f (A) A E l w,e-" ,. d d d h d h =-Z " (A)-f(A)-f(A)-Z(A)